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(57) Abstract: The invention relates to a method of optimizing interpreted programs, in a virtual machine interpreter of a bytecode- 
based language, comprising means for dynamically reconfiguring said virtual machine with macro operation codes by replacing an 

original sequence of simple operation codes with a new sequence of said macro operation codes. The virtual machine int^preter 

is coded as an indirect threading interpreter thanks to a translation table containing the implementation addresses of the operation 
codes for translating the bytecodes into the operation codecs implementation addresses. Application: embedded systems using any 

^ bytecode-based programming language, set to box for interactive video transmissions. 
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FIELD OF THE INVENTION 

-n.. invention relates to nm-time optimization of interpreted progi^- It 
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„,3.es. n-orc panicuUHy. .o a n.efl»d for optoizing in.crp«.e<> programs --"f " 
^ n«to= wMch dynanucally r^n^^ i«c.f wi* new »aon. opera«o„ cod«. The 
mvenSon applies to any bytecode-ba^ piograranring language. 

BACKGROUND OF THE INVENTION 

By«code-based languages p^grammer-visible stacks are popular as 
i^ediate languages for oompUcrs. and aUo as nuchine-indepcndent executabU progran, 
mentations. Tltey offer significant advantages for netwoA convputmg. "n-e ^clc 
l^zingdir.c.«^edcodebys.lecUvein-lining^byl.Piunuu,«>andF.I.cc^^^^ 

pLedings of the ACM SIGPl^ '98 Coni^ence on ProgramnUng Language Des.gn and 
^,:!:Lon CPLDI,, Monttcal, Canada, lune .7. 199S. pp.291.300, describes «e 
as lentioned in the opening paragraph, for optinnzing interpreted progran.. A vtrn^ 
^hine (VM) is used to interpret the progran. thanks to a VM ^terpreter. Tl^ VM .s a 

software implementad"n«P--*8--"'=^°"'^^°°^"'°'' ,^, 
appUcaaons especially compiled for «.s archite«ure are execu«d. Tlte ---- j'*^ 
^p,ocessor/n«chin.arecalledbyt=code.Tl.eVMinte,re.««.hepar.of.heVM 

.hich represents .hebytecodes-executionmectausm.Tl>ebytecodes are sa.d.0 be 

in.erpr^byfl.e VMinterpreter.Thebytecod.s' execution mechanism is currmtfe. 
Clentedasan infinite loop wi.haswi.chcasebloc.Th..echm,uedescribed mthec..ed 

ardcle applies to direct ftreaded in.en>r«ers. Traded code interpreters execute to 
hytecodes in line. Each bytecode translation contains the ref^ence to the next bytecode. 
^fore, .he bytecode translation as executed by a threaded interpreter does no. mvolve the 
^finite loop. Even though threaded interpreters offer a pertonnance advanuge. th^ are too 
slow and require too much memoty to be convenient for most embedded systems. In a d.rec. 
U^ed code interpreter, as described in the cited article, .he VM bytecodes are represented 
wifl, *e address of their implemenUtion, so that each bytecode can directiy jump to to 
implementation of to next by.ecode. A table is initialized before to translation operanon. 
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with the addresses of each bytecode of the application in order that, when the bytecode 
translation takes place, the physical addresses of the bytecode implementations can be 
quickly accessible. The table allows to switch firom a bytecode to another one. Direct 
threaded interpreters are rather fast but they involve code expansion. By changing bytecodes 
into direct threaded codes, the code size is increased by approximately 1 50%, because the 
operation codes are replaces with the addresses of their implementation code. In general, 
addresses need 4 bytes whereas the operation codes need only 1 byte. Therefore, direct 
threaded interpreters increase memory consumption and are thus not very suitable for 
embedded systems. 



SUMMARY OF THE INVENTION 

It is an object of the invention to provide a method for optimizing run-time of 
interpreted programs which is very convenient for embedded systems. Such systems may be, 
for example, satellite or cable transmission systems embedded into a digital video receiver, 
often called a set top box. But the invention also applies to any product whose operating 
system is based on a bytecode-based programming language. The invention also allows to 
save memory and CPU resources and can improve the performance of the system. 

In accordance with the invention, it is described a method of optimizing 
interpreted programs in a virtual machine interpreter of a bytecode-based language, wherein 
the virtual machine dynamically reconfigures itself with new macro bytecodes (or opcodes) 
replacing sequences of simple bytecodes, and wherein the virtual machine interpreter is 
coded as a threaded code interpreter for translating the bytecodes into their implementation 
codes. The threaded code interpreter according to the invention is coded as an indirect 
threaded code interpreter thanks to a reference table which contains the implementation 
addresses of the bjdecodes in order that during translation of a bytecode, the address of the 
next bj^ecode is retrieved to be able to jump to the next bytecode. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The invention and additional features, which may be optionally used to 
implement the invention, are apparent from and will be elucidated with reference to the 
drawings described hereinafter. 

Fig. 1 is a bloc diagram illustrating the features of a method according to the 

invention. 
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Fig. 2 is a bloc diagram illustrating the features of a method according to the 
preferred embodiment of the invention. 

Fig- 3 is a schematic diagram illustrating an example of areceiver accordmg to 

the invention. 

DETAILED DESCRIPTION OF THE INVENTION 

^ an example, to iU«s«a.e a novel run-ti»e cpUnn.a.ion s«.egy appUcabl. u> any byucode- 

'^";Heappr„ach„onnany.a.enbyIus..In-Tin>.(nT)oon^i,e«is»^ 
aMogeU^er tt.e Java virtual machine (VM) interpreter and to tramlate the applioafon's 
.ytecode i„«. nadve machine c«le priot to its exertion (hence .he Inst-In-Time 
lamination. This process involves m^letstanding the original '^^'^l^ '^ 
U into a more convenient native fonn. While this may he an efficren. way of 
aJng performance, it also comes a. the expense of a very lar^ memory » 
U.e one hand, hecause a hytecode-hased language is more compact than a nanve code, a.^ of 
targe CPU (Central Processing Unit) resources in .he other hand, because re^mapprng Java 
bytecodes on the target machine is not an easy task. 

The invention is also based on some sort of dynamic code generation, but .ts 
, goal is not that of translating the application's Java by«cod= into native machine c^ but 
^ to dynamicaUy »lapt tire Java VM to tire execution of .he appUcation's specie 
.yrecode sequences. The original appUcation. Java byecode is thus preserved, whrle ti>e VM 
is dynamically e„rich«l with novel bytecodes or operation codes (opcodes) mrprovmg rU 

execution efficiency. 
5 There are several advantages to this approach : 

Itdoesnotincreasethesizeoftheexecutablecode:theapplicationisleftintomemory- 

efScient Java's byttcoded representation. 

The VM's execution mechanisms is economic : tttere is only one execution 
nrechanism, titerefore the VM executing .he appUcation will not have «. deal witi. multiple 
code representations which contributes to reduce its si« and improve its rehabrhty. 

The code generation technique is rattier simple: VM optimizer has a very 
simple stiucti^e. tire application's byrecode ^alysis is a one-pass tablc^Wven procedure 
.aldng very linle CPU resources, and which directly drives the synttresis of new bytecodes. 
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These properties make the invention suitable for embedded applications. The 
foundation of the optimization technique according to the invention lies in the study of the 
costs of the very basic mechanisms of an interpreter with respect to a category of "typical" 
appUcations. The relevance of the application's profile lies in the potential benefit attainable 
5 firom the various optimization techniques that might be envisaged. Since the target is 
embedded applications, what might be define as "typical" appUcations are, for example, 
control applications, graphical user interfaces, and so forth. 

It is assumed that the target appUcations are weU mapped on the primitives 
offered by the underlying VM (object manipulations). Therefore, they will not benefit much 
1 0 fi-om radical code transformations, but rather firom a general improvement of the VM's 

execution mechanisms. To understand how to improve the efficiency of the VM, it was made 
use of Amdhal's law. In the version stated by Hemiessy and Patterson, Amdhal's law is 
expressed as follows : "the perforaiance improvement to be gained firom using some faster 
mode of execution is limited by the firaction of time the faster mode can be used", or more 
15 synthetically : "make the common case fast". 

Interpreter's perforaiance depend on the representation chosen for executable 
code and on the mechanism used to dispatch the bytecodes. The first approach to reduce the 
implementation cost was to reduce the cost of instruction dispatching because the heart of an 
interpreter is its instmction dispatching mechanism. The typical interpreter, called pure 
20 bytecode interpreter, is implemented like a processor simulation : a large switch statement 
sitting in an endless loop, dispatching instructions to their implementations. Therefore, the 
inner loop of a piu-e bytecode interpreter is very simple : fetch the next bytecode and dispatch 
to the implementation using a switch statement. The interpreter is an infinite loop containing 
a switch statement to dispatch successive bytecodes, and passes control to the next bytecode 
25 by breaking out of the switch to pass control back to the start of the infinite loop. The 

following set of instmctions illustrates an implementation of a typical bytecode interpreter. 



Loop ( 

Op = *pc++; 
30 Switch (op) { 

Case op_l : 

// op_l's implementation 

break; 
case op_2 : 
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/ / op_2 ' s implementation 
break ; 

case op_3 : 

// op_3's implementation 

break; 



Lsuming the compiler optimizes the jump chains from the breaks through the 
implicit jump at the end of the loop back to its begimiing, the overheads associated with this 

approach are as follows : 

increment the instruction pointer pc, 

fetch the next bytecode from memory, 

a redmdant range check on the argument to switch, 

fetch the address of the destination case label from a table, 

jump to that address, 

and at the end of each bytecode : 

jump back to the start of the loop to fetch the next bytecode. 

In this case the cost of instruction dispatching, ignoring all other sources of 
inefficiency such as the actual implementation of the switch statement, consists of : 

2 memory accesses : one to retrieve the value of the next instruction, one to 
retrieve the address of the instruction's implementation, 

plus 2 branches : one to jump to the bytecode's implementation and another 
one to go back to the begimiing of the loop. Jumps are among the most expensive instiiictions 

on modem architectures. 

Pure bytecode interpreters are easy to write and to understand. They are also 
highly portable but rather slow. They are thus not convenient for embedded systems. In the 
case where most bytecodes perform simple operations, as in the example illustrated herein 
before, most of tiie execution time is wasted in performing the dispatch. Actiially, in order to 
be aware of tiie real cost of tiie mechanism, it should be compared witii tiie cost of the 
execution of a single bytecode. Java bytecodes have a very low-level semantics, and tiieir 
implementation is often trivial. Therefore, the most commonly executed bytecodes are 
actiially less expensive than the dispatching mechanism itself. 

A first improvement in efficiency according to the invention is the adoption of 
indirect threaded code as illustrated witii tiie set of instructions below : 
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Op_l_lbl : 

// op_l's implementation 
goto opcode_table (*pc++) ; 

Op_2_lbl : 

5 // op_2's implementation 

goto opcode_table (*pc++) ; 

Op__3_lbl : 

// op__3's implementation 
goto opcode_table (*pc++) ; 

10 where Op_l_ Ibl, Op_2_ Ibl and Op_3_ Ibl represent 3 different operation codes interpreted 
by the VM interpreter. 

According to this implementation, called indirect threaded code, the VM is 
coded as an indirect threaded code interpreter. During bytecode translation, the address of the 
next bytecode is resolved. A reference table, denoted opcode_table, contains the bytecodes 

15 implementation addresses. The reference table is accessed by an index of a pointer (*pc-f-f). 
For each bytecode translation, the address of the next bytecode is retrieved to jiraip to the 
next bytecode. In this way each bytecode implementation directly jumps to the next bytecode 
implementation, we have saved one branch, the outer loop, and the uimecessary inefficiency 
of the switch statement's implementation (range checking and default case handling). 

20 According to a preferred embodiment of the invention, the translation is 

earned out by exploiting unused bytecodes of the b3^ecode-based language VM specification. 

The bloc diagram of figure 1 summarizes the mains steps of the method 
according to the invention for translating a bytecode, e.g. the bytecode bipush, into native 
instructions with an indirect threaded code interpreter : 

25 step K0= BIPUSH ; begiiming of the method of translating the bytecode 

bipush which consists of putting word on a stack, the Vi word being the bipush parameter 
(par) 

step Kl= PAR ; retrieve the bipush parameter (par) 

step K2= PUT; put the bipush parameter on the stack 
30 step K3 = GOTO; go to the next bytecode (goto opcode__table (*pc)) by 

looking into a reference table opcode_table containing the address of the next bytecode' s 
implementation. 

The adoption of threaded code by itself can double the VM's performance, but 
as we will see in the following it can also offer other interesting optimization opportunities. A 
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s^tistic analysis of lava's bytccodes shows thai, on avarage. about .very 5-6 insm-ctions 
Aere is a branch. On any modem CPU. branches are intrinsicdly expensive insBuchons. 
rinc. they can cause pipeline stalls and/or trigger external bus activity. Besides, for loop 
filing or method call in-lining, there is no, much «.a, can really be done abom tt. Ey«. 
when r^ompiling the code into a native represeniation, the control statemems wiU strU be 

Recent sttidies on the CPU utilization for object oriented applications on high- 
end workstations show that the CPU can sp«.d as much as 70% of its clock cycles to recov^ 
fion, pipeline stalls, as the effect of mispredicted branch statements, and to wait for dau^d 
i^cns ftom amainmemory (cache misses). Additionally. CPUs available in embedded 
systems normally have very small caches, no haniware assistance for dynamic branch 
prediction, and low and/or narrow memory interfaces witir no caches. These addruonal 
constraints will reduce even further the CPU utilization and perfonnance. 
Java bytecodes can be separated into two categories : 
simple operation codes Ooads. stores, arithmetic and control statements) and 
complex operation codes (memory management, synchronization, etc.). 
Simple bytecodes are typically less expensive than die dispatching 
mechanism. Complex bytecodes are instead much more expensive, the dispatching cos. 
representing only a minimal ftaction of tire total cost of ttiebytecode execution cost Smtple 
, bytecodes are also executed much more frequentiy (about an order of magnimde) than 
complex one^ implying tt.at a classical Java interpreter spends most of its time dispatctang 
bytecodes rather ««n really doing anytinng useful. It is tims ass^ned that it would be 
definitively more effective to reduce the dispatching cost for simple bytecodes fl«n for 
complex ones. 

5 Translating bytecodes into indirect threaded code also gives fl>e oppormmty to 

^e arbitrary transfomutions on the executable code. One such transformation is to detect 
common sequences of bytecodes and translate them into a single threaded "macro code 
This macro code perfom.s die wo* of the entire sequence of original bytecodes. Therefore, 
according to a preferred embodimem : f the invention, it is proposed to replace sequences of 
,0 simple bytecodes by some equivalent "macro codes". For example, as presented in tite cted 
article flte bytecodes "push literal, push variable, add, store variable" can be translated in«, a 
smgle "add-literal-to-variable" macro code in tite indirect dreaded code. Such optimization 
are effective because fltey avoid die overhead of ttte multiple dispatches ti>at are implied by 
a,e original bytecodes, but elided wiflun «.e macro code. A single macro code which .s 
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translated from a sequence of N original bytecodes avoids N-1 bytecode dispatches at 
execution time. More details about how to build macro codes can be found in the cited 
article. Such macro codes will have to satisfy the following criteria : 

Macros have to be made out of sequences of simple bytecodes, since there is 
5 no point in reducing the dispatching cost of complex ones. 

Macros must not contain instructions that are possible branch targets, 
otherwise one would have to radically change the VM execution mechanism. A macro itself 
can be a branch target. 

Macros must terminate with control statements or method calls, since the cost 
10 of a native branch is equivalent to that of a dispatch operation. 

For implementation simpUcity, the maximal length of a macro should be 
approximately of 15 bytecodes. The "natural" average macro length being of 4-5 bytecodes. 
From these criteria it is very simple to construct such macro sequences, taking very httle -and 
bounded- CPU time. A simple scan of a method's bytecode is indeed enough, and most of the 
15 parsing can be table driven and single-bytecode based. 

According to a particular alternative of the preferred embodiment, which takes 
into accoimt that xmused bytecodes are very few (30-40 on average) a two-byte representation 
can be used for the new bytecodes r^resenting the new macro-instruction. The operands of 
the original sequence are grouped right after the new sequence, which leaves them easily 
20 accessible by incrementing the program counter of the virtual machine. 

Once a process is scanned, macros can be constmcted by simply cutting and 
pasting together the binary code produced by the compiler for the threaded code interpreter. 
Macros are just considered as normal bytecodes by the threading dispatcher. 

Figure 2 sununarizes the preferred embodiment of a virtual machine according 
25 to the invention. The VM is implemented to load programs containing bytes codes to be 
interpreted by the VM interpreter. The main steps of the method are the following : 

step K0= INIT: initialization of the procedure executed by the VM by loading 
the programs containing the bytecodes , 

step Kl= OPCODE : to retrieve the bytecodes to be interpreted, 
30 step K2= MACRO : replacement of sequences of simple bytecodes with macro 

bytecodes, 

step K3= TRANS : interpretation of the macro bytecodes using the indirect 
threaded interpreter method as described in figure 1, 

step K4= RES : get the result, end of the method. 
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SuttsUca.analysisperfonnedoae.=cutio„»ac.sofac«-Javaapp.i03«ons. 
1 of 4-5 bvtecodes, and that, after the code 

^„™a«on.n«cro.-be«^^up^^^^^ ^^^^^^^^^ 

rr:rr.:isi.o«— ----- 

Kocionificantlv reduced by using the mvenuon. 

0 i^smicoons can also be «du«d by 

si^fica^ly unp^ved. T^e ac«^ ^ ^^ to me cos. of a cache line 

^„,.pipel.esu..^»~-^^^^ 
ffll.On-o.en.orycba..«^«l^^ ^ 

::iirrr:iTrrr::ieco.aa.— ...^^ 

which are left with. sequences of 

One ofthe advantages of macros IS tnaimcym e , . 

20 Oneoimc e »^„ences can be found elsewhere in the 

r:::^ -rr..*a.a....can.pa«o..cn^c.. 
wL^gin«>aocoun.*.reusefacU,r,*ememo.y<bo*rin.usedbyn,ac,ocode 
by Blang n.«. ac ^ ^^^^ ^ 

BkelyUisnotworthtradingagainstftedoublingofmemoryfooWnnt. 
,„ ^„*eradva„Ugeofn,ac«.sis.ha.U,eyaono.haveanyunpa«o„U.e 

no.a.by.ecoae.spa.cMng— — ^ 

compiled and non-compiled processes ana n 
native code interfaces. 
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Object-oriented languages like Java are characterized by the presence of very 
small units of code, Java processes are also very hard to inline^ since they are almost always 
potentially polymorphic. Therefore, even if a fully optimizing compiler would be able to 
better map the process execution semantics on the underlying processor architecture, the 
5 overhead of the preamble and conclusion of binary translated processes would often suppress 
any advantage. 

To improve execution efficiency, a stack catching technique can be used, 
which keeps the first three locations of the Java stack inside the processor's register file, 
reducing considerably the number of memory accesses. The technique exploits the fact that 

10 the target processor is a stack machine itself. The original bjrtecode implementations are 
substituted with equivalent processor instmction sequences. By using a trivial translation 
table and a simple cost function (number of memory references), very fast and efficient 
compilation technique can be achieved. The cost reduction of memory Input / output will 
now be described, in the case of Java as an example, according to another alternative 

1 5 embodiment of the invention. 

Java is a stack-based language: bytecodes conmiunicate with each other using 
memory. Every single bytecode execution implies at least one memory access, which turns 
out to be very expensive. Considering, for instance, the following simple expression : 
C = a + b; 

20 In a stack based language it is translated into : 



Push a — 1 read, 1 write 

Push b —1 read, 1 write 

Add — 2 read, 1 write 

Store c — 1 read, 1 write 



25 which represents nine memory access operations. A CPU with a minimum of internal state 
can do the same with only three memory accesses. Considering the fact that on a modem 
processor architecture, memory references are among the most expensive operations, it is an 
ideal field of optimization. With a little additional coding effort, a version of the Java 
bytecodes can be made to exchange data through machines registers instead than through 

30 external memory. Macros can then be created, starting fl-om these specialized bytecodes 

which are called strands, reducing the number of memory accesses within a macro by more 
than a factor of two. 

An implementation of the "macroizer" and of the bytecode "standifier" would 
not need too many lines of code. Partial rewrite of the interpreter's loop can be estimated, for 
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*^°''^"*;IlTeas.esof«,en^ti».ebavebeennra<.ew..eh.^-..a.ein» 

• ,n for interactive video transndssion.Kcompnses a decoder, e.g. 
.e,.opboxrece.v«20f«m^«^^^^ ^^^^^ 

^a«ble wtth ^ ,^3^^ 23 an encoded signal ftom a 

"-■^""•^'T^^t^^.ter.eivedsigna.inorder.or^rievethe.ransn.^d 
viaeo transnnrter 24 and * ^ box can be efBci««.y 

aata «> be displayed on a vtdeo ^'^'^^^ ^ ^ Java in 

software implemented using a syst«n that executes an ^ ^ 

•~::c::^rir:»-p~^essorcPu 

^ software code '>°*°"^7'^="^^^^^„„ ^ in figure 1 or X 

'"--°"'"r::gT:rrren.ofd.inven,io...eset.op.x2„can 

According . ^ part of the received signal. In this case, the 



to 

distant sender. 
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1 A method of optiini2dng interpreted programs in a virtual machine interpreter 

of a bytecode-based language, wherein the virtual machine dynamically reconfigures itself by 
replacing an original sequence of simple bytecodes with a new sequence of macro bytecodes 
and wherein the virtual machine interpreter is coded as a threaded code interpreter for 
translating the bytecodes into their implementation code, comprising a reference table which 
contains references to the addresses of the implementation of the bytecodes m order that 
during translation of the current bytecode, the address of the implementation of the next 
bytecode is retrieved to be able to jump to the next bytecode. 

2, A method according to claim 1 , whaein the bytecodes of the original 

sequence are grouped after the new sequence of said macro operation codes. 



3 A method accorxiing to any of claims 1 or 2, wherein the virtual machine 
mteipreter comprises a predetermined set of bytecodes, some of which are unused, and 

1 5 wherein said new sequence of macro operation codes is implemented by exploiting said 
unused bytecodes. 

4 A method according to claim 3, wherein the unused bytecodes are encoded 
with at least a two-byte representation. 

20 

5 A method of optimizing interpreted programs, in a virtual machine for a 
bytecode-based language, comprising the following steps : 

initiaUzation by loading programs containing the bytecodes, 
replacement of sequences of simple bytecodes with macro codes, 
25 interpretation of the macro bytecodes using an indirect threaded interpreter for 

translating the bytecodes into their implementation code, comprising a reference table which 
contains references to tiie addresses of tiie implementation of the bytecodes in order that 
during interpretation of the current bytecode, the address of the implementation of the next 
bytecode is retrieved to be able to jump to the next bytecode. 
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, A con^uter p«>gr3m p»duc. for b«ng loaded a memory. -^^^^ 

!r^.„.ro.C3>^.ap™.ooa„yo«....c««.— ...n.oneofc,a.n. 

lto5. 

A receWer for receiving transmission signals, the receiver comprising a 
<,^WCPU)andamemory(MEM)forstoringsoftwarecodeportionsrepresentmg 
processor (CPU) and tn ry , 
instructions for causing the processor to carry 

1 to 5. 

Amethodofmaldng.vaUablefordov™loadingaoon.p««rp,ogram 
LpHsi.gi.s™.onsfo.«ecu.^*=»-oda,cU^ina„yoncof*ecU....»^^ 

into a receiver as claimed in claim 7. 
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